Many photonic integrated circuits (PICs) include mode converters that transfer light between different device layers from one waveguide to another. For example, PICs otherwise implemented in silicon may utilize, at the input(s) and/or output(s) of the PIC, silicon-nitride-based grating couplers because they can operate across a wider temperature range than silicon-based couplers. In this case, the mode converter serves to couple light from a silicon waveguide in the silicon device layer into a silicon-nitride waveguide disposed above the silicon waveguide and coupled to the silicon-nitride grating coupler (or vice versa). In the coupling region, the silicon waveguide is tapered down to a small cross section. Over a large portion of the taper, light is therefore highly confined to a small cross-sectional area, which makes the light more sensitive to any surface roughness. The relatively high surface roughness achieved with conventional manufacturing techniques tends to result in a correspondingly high scattering loss from this portion of the taper. The high scattering loss, in turn, contributes to a higher insertion loss for the device, and the backscattered portion of the scattered light contributes to a higher return loss.